{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preface\n",
    "\n",
    "The following examples are what I wrote when learning shell. Some are typical. Refresh it when necessary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 模式匹配和位置参数\n",
    "\n",
    "$#：参数个数\n",
    "\n",
    "$*：将所有命令行视为单个字符串（分隔以$IFS的第一个字符为准，一般为空格，tab，换行）\n",
    "\n",
    "$@视为单独字符串（适合保留单个参数中的空白部分）\n",
    "\n",
    "双引号会确切地处理括起来文字中的转义字符和变量、算术、命令替换等；而单引号都看作字面上的意义!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/cshi/long.file.name\n",
      "cshi/long.file.name\n",
      "long.file.name\n",
      "/home/cshi/long.file\n",
      "/home/cshi/long\n",
      "25\n",
      "there are 3 total arguments\n",
      "bash: there: command not found\n",
      "$*-------------\n",
      "i is hi\n",
      "i is there\n",
      "i is world\n",
      "$@-------------\n",
      "i is hi\n",
      "i is there\n",
      "i is world\n",
      "quoted $*-------------\n",
      "i is hi there world\n",
      "quoted *-------------\n",
      "i is hi there\n",
      "i is world\n"
     ]
    }
   ],
   "source": [
    "path=/home/cshi/long.file.name\n",
    "echo $path\n",
    "echo ${path#/*/} #从开头匹配最短的，返回剩下的\n",
    "echo ${path##/*/} #从开头匹配最长的，返回剩下的\n",
    "echo ${path%.*} #从结尾匹配最短的，返回剩下的\n",
    "echo ${path%%.*} #从结尾匹配最长的，返回剩下的\n",
    "echo ${#path} #返回字符长度\n",
    "\n",
    "# 如果给定参数一，且非空，则取其值，否则取/dev/tty.诸如类似的还可将-换成 =,+,?\n",
    "filename=${1:-/dev/tty}\n",
    "\n",
    "set -- hello \"hi there\" world #设置位置参数\n",
    "echo there are $# total arguments\n",
    "\n",
    "echo '$*-------------'\n",
    "for i in $*; do echo i is $i;done\n",
    "echo '$@-------------'\n",
    "for i in $@;do echo i is $i;done  # 没有双引号时，$*与$@一样\n",
    "echo 'quoted $*-------------'\n",
    "for i in \"$*\";do echo i is $i;done #加了引号后，$*表示一个字符串\n",
    "echo 'quoted *-------------'\n",
    "for i in \"$@\";do echo i is $i;done #加了引号后，$@保留了空白，此种方式用得多"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# []和test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "if [ -f \"$file\" ]：为一般文件\n",
    "-s file：文件非空\n",
    "\n",
    "-n string：字符串非null\n",
    "-z string：字符串为空\n",
    "n1 -lt n2：n1小于等于n2\n",
    "\n",
    "if [ -f \"$file\" ] && ! [ -w \"$file\" ]：如果是文件且不可写（一定要加引号）\n",
    "\n",
    "if [ \"X$1\" =  \"X-f\" ]; then :    为了避免-号干扰到test命令，前面加了前置字母，且加个:号表示不做任何事\n",
    "\n",
    "if [ $1 = \"create\" ]; then，如果这样写，要注意一旦$1为空，则表达式错误，报错：\n",
    "> [: =: unary operator expected"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 判断、循环与函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## if"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cshi:x:1002:1002::/home/cshi:/bin/bash\n",
      "not found\n",
      "equal\n"
     ]
    }
   ],
   "source": [
    " # !写在前面\n",
    "if ! grep 'cshi' /etc/passwd; then #如果有必要，则重定位标准输出和错误\n",
    "     echo \"not found\"\n",
    "fi\n",
    "if ! grep 'not-exist' /etc/passwd; then\n",
    "    echo \"not found\"\n",
    "fi\n",
    "\n",
    "a='a'\n",
    "b='b'\n",
    "if [ $a=$b ]; then echo equal;fi\n",
    "if [ $a = $b ]; then echo equal2;fi #一定要加空格！！！是比较不是赋值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## for/while/until"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 计算从1加到100\n",
    "i=1  #等号左右不能加空格！！不能再像c、python一样加空格了！\n",
    "sum=0\n",
    "\n",
    "while [ \"$i\" -le 10 ]\n",
    "do\n",
    " sum=$(($sum+$i)) \n",
    " i=$((++i)) #shell算术运算符都位于$(())内\n",
    "done\n",
    "\n",
    "echo $sum\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 行读取文件\n",
    "printf \"*************************************\\n\"\n",
    "echo \" cat file while read line\"\n",
    "cat test.txt | while read line\n",
    "do\n",
    "  echo $line;\n",
    "done\n",
    "\n",
    "printf \"*************************************\\n\"\n",
    "echo \"while read line <file\"\n",
    "while read line\n",
    "do\n",
    "  echo $line;\n",
    "done < test.txt\n",
    "\n",
    "printf \"*************************************\\n\"\n",
    "echo \"for line in cat test.txt\"\n",
    "SAVEIFS=$IFS\n",
    "IFS=$(echo -en \"\\n\")\n",
    "for line in $(cat test.txt)\n",
    "do\n",
    "  echo  $line;\n",
    "done\n",
    "IFS=$SAVEIFS\n",
    "#当文件中有空格或者tab 时，一定要设置一下IFS变量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## case\n",
    "\n",
    "> 结构：\n",
    "\n",
    "```\n",
    "case $1 in\n",
    "-f)\n",
    "    ...\n",
    "    ;;\n",
    "*)\n",
    "    ;;\n",
    "esac\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 每30秒检查特定用户是否登陆\n",
    "check() {\n",
    "    while true; do \n",
    "        if who | grep cshi > /dev/null; then \n",
    "            break;\n",
    "        fi\n",
    "        sleep 30\n",
    "    done\n",
    "    echo \"done\"\n",
    "}\n",
    "\n",
    "wait_for_user() {\n",
    "     until who | grep cshi >/dev/null #until的条件为假时，继续执行\n",
    "     do\n",
    "      sleep ${2:-30}\n",
    "     done\n",
    " }\n",
    " \n",
    " #父脚本参数会暂时被函数参数所隐藏，即$1,$2指的是函数参数，$0仍为脚本名\n",
    " #wait_for_user cshi 3 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# grep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ababcd\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "# 后向引用，最多可包括9\n",
    "echo 'ababcd' | grep '\\(ab\\).\\1'\n",
    "echo 'ababcd' | grep '\\(ab\\).*\\1'\n",
    "\n",
    "echo abbba | grep 'ab\\{4,\\}' #至少出现几次\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#去掉空行，注意不适合dos格式的文件，含有^M\n",
    "grep -v '^$' a.txt > tmp "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sed\n",
    "\n",
    "> 无-n有-p时，打印所有行，且匹配行会多打印\n",
    "有n无p时，不打印\n",
    "有n有p时，仅打印匹配行，这是最为常用的！！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sed -n '$p' a.txt  #打印最后一行\n",
    "sed -n '1,10p' a.txt #打印1-10行\n",
    "sed -n '2,$p' a.txt >tmp #相当于删除民第一行\n",
    "sed '/foo/,/bar/ s/a/A/g' a.txt #从含有foo的行开始到含有bar的行结束，a换成A\n",
    "sed '/bc/!s/d/D/' a.txt #将没有bc的行的所有d换成D，即!放在正则后面表示否定\n",
    "sed -i 's/s/S/' #直接将文件a中的s换成S!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#unix转dos\n",
    "sed 's/$/\\r/' unix.txt >dos.txt\n",
    "\n",
    "#dos转unix\n",
    "sed 's/.$//' dos.txt >unix.txt "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## 打印匹配行与行号\n",
      "my name is cshi\n",
      "1\n",
      "Acba\n",
      "abcA dcba\n",
      "abcA Acba\n",
      "Babc\n",
      "BaBcB\n",
      "/home/cshi/\n",
      "oprofile:x:16:16:Special user account to be used by OProfile:/home/oprofile:/sbin/nologin\n",
      "iqiyioops:x:1001:1001:iqiyioops Puppet-managed User:/home/iqiyioops:/bin/bash\n",
      "cshi:x:1002:1002::/home/cshi:/bin/bash\n",
      "test:x:1003:1003::/home/test:/bin/bash\n"
     ]
    }
   ],
   "source": [
    "cat << eof >tmp\n",
    "my name is cshi\n",
    "yes i can\n",
    "eof\n",
    "#cat tmp\n",
    "echo \"## 打印匹配行与行号\"\n",
    "sed -n -e '/my/p' -e '/my/=' tmp\n",
    "\n",
    "echo 'abcd dcba' | sed 's/a.*d/A/' #默认匹配最长的串\n",
    "echo 'abcd dcba' | sed 's/a*d/A/' #可以匹配0个a\n",
    "echo 'abcd dcba' | sed 's/a*d/A/g'\n",
    "\n",
    "echo abc | sed 's/b*/B/'\n",
    "echo abc | sed 's/b*/B/g'\n",
    "\n",
    "#后向引用\n",
    "echo /home/cshi/ | sed 's;\\(/home\\)cshi/;\\1/cs/;' #以;作为分隔符\n",
    "\n",
    "# 打印含有/home的帐号\n",
    "cat /etc/passwd | sed -n '/\\/home/p' # 要转义/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# cut"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "grafana:/usr/share/grafana\n",
      "logstash:/opt/logstash\n",
      "sphinx:/var/lib/sphinx\n",
      "redis:/var/lib/redis\n",
      "hacluster:/var/lib/heartbeat/cores/hacluster\n",
      "ricci:/var/lib/ricci\n",
      "nrpe:/var/run/nrpe\n",
      "telegraf:/etc/telegraf\n",
      "test:/home/test\n",
      "cloud-agent:/opt/cloud-agent/agent\n"
     ]
    }
   ],
   "source": [
    "# -d表示分隔符，默认\\t，-f表示字段\n",
    "cut -d: -f 1,6 /etc/passwd | tail "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# sort"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dbus:x:81:81:System message bus:/:/sbin/nologin\n",
      "daemon:x:2:2:daemon:/sbin:/sbin/nologin\n",
      "cshi:x:1002:1002::/home/cshi:/bin/bash\n",
      "couchbase:x:496:496:Couchbase system user:/opt/couchbase:/sbin/nologin\n",
      "cloud-agent:x:850:850:Cloud-Agent agent:/opt/cloud-agent/agent:/bin/bash\n",
      "bitlbee:x:489:488:BitlBee User:/var/lib/bitlbee:/sbin/nologin\n",
      "bin:x:1:1:bin:/bin:/sbin/nologin\n",
      "apache:x:48:48:Apache:/var/www:/sbin/nologin\n",
      "adm:x:3:4:adm:/var/adm:/sbin/nologin\n",
      "abrt:x:173:173::/etc/abrt:/sbin/nologin\n",
      "## sorted by number\n",
      "uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin\n",
      "mail:x:8:12:mail:/var/spool/mail:/sbin/nologin\n",
      "halt:x:7:0:halt:/sbin:/sbin/halt\n",
      "shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown\n",
      "sync:x:5:0:sync:/sbin:/bin/sync\n",
      "lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin\n",
      "adm:x:3:4:adm:/var/adm:/sbin/nologin\n",
      "daemon:x:2:2:daemon:/sbin:/sbin/nologin\n",
      "bin:x:1:1:bin:/bin:/sbin/nologin\n",
      "root:x:0:0:root:/root:/bin/bash\n"
     ]
    }
   ],
   "source": [
    "# t限定分隔符(注意与cut分隔符的区别)，k指定字段\n",
    "sort -t: -k1,1 -r /etc/passwd | tail\n",
    "\n",
    "echo \"## sorted by number\"\n",
    "# n 以数值排，r反序\n",
    "sort -t: -k3nr /etc/passwd | tail"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# awk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "user grafana is readlly grafana user\n",
      "user logstash is readlly logstash\n",
      "user sphinx is readlly Sphinx Search\n",
      "user redis is readlly Redis Server\n",
      "user hacluster is readlly heartbeat user\n",
      "user ricci is readlly ricci daemon user\n",
      "user nrpe is readlly NRPE user for the NRPE service\n",
      "user telegraf is readlly \n",
      "user test is readlly \n",
      "user cloud-agent is readlly Cloud-Agent agent\n"
     ]
    }
   ],
   "source": [
    "# 默认以空白分隔字段，且不要忘记中间的逗号，否则会连接相信的所有值！  输出以空格为分界\n",
    "awk -F: '{print \"user\",$1,\"is readlly\",$5}' /etc/passwd | tail"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 数制处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12\n",
      "FF\n",
      "10\n",
      "5\n",
      "(()):10,10,255,255,15\n"
     ]
    }
   ],
   "source": [
    "echo \"obase=8;10\" | bc\n",
    "echo \"obase=16;255\" | bc\n",
    "let num=012 #八进制\n",
    "echo $num\n",
    "let num=3#12 # 3进制\n",
    "echo \"$num\"\n",
    "\n",
    "((num=012))\n",
    "((num2=8#12))\n",
    "((num3=0xff))\n",
    "((num4=16#ff))\n",
    "((num5=2#00001111))\n",
    "echo \"(()):$num,$num2,$num3,$num4,$num5\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# subShell和代码块\n",
    "\n",
    "subshell:以()包裹，在另外的进程中执行，不改变主脚本目录\n",
    "tar –cf . | (cd /newdir; tar –xpf .)\n",
    " \n",
    "代码块：以{}包裹，不会建立新进程，且会对主脚本状态造成影响。{}在shell中为关键字，所以}的位置要单独成行\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# misc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "11\n",
      "3\n",
      "root,/root\n",
      "bin,/bin\n",
      "daemon,/sbin\n",
      "adm,/var/adm\n",
      "lp,/var/spool/lpd\n",
      "sync,/sbin\n",
      "shutdown,/sbin\n",
      "halt,/sbin\n",
      "mail,/var/spool/mail\n",
      "uucp,/var/spool/uucp\n",
      "operator,/root\n",
      "games,/usr/games\n",
      "gopher,/var/gopher\n",
      "ftp,/var/ftp\n",
      "nobody,/\n",
      "dbus,/\n",
      "vcsa,/dev\n",
      "rpc,/var/cache/rpcbind\n",
      "abrt,/etc/abrt\n",
      "apache,/var/www\n",
      "haldaemon,/\n",
      "ntp,/etc/ntp\n",
      "saslauth,/var/empty/saslauth\n",
      "postfix,/var/spool/postfix\n",
      "tomcat,/usr/share/tomcat6\n",
      "webalizer,/var/www/usage\n",
      "sshd,/var/empty/sshd\n",
      "mysql,/var/lib/mysql\n",
      "tcpdump,/\n",
      "oprofile,/home/oprofile\n",
      "puppet,/var/lib/puppet\n",
      "iqiyioops,/home/iqiyioops\n",
      "couchbase,/opt/couchbase\n",
      "nagios,/var/spool/nagios\n",
      "exim,/var/spool/exim\n",
      "cshi,/home/cshi\n",
      "rabbitmq,/var/lib/rabbitmq\n",
      "nginx,/var/lib/nginx\n",
      "memcached,/var/run/memcached\n",
      "dockerroot,/var/lib/docker\n",
      "rpcuser,/var/lib/nfs\n",
      "nfsnobody,/var/lib/nfs\n",
      "bitlbee,/var/lib/bitlbee\n",
      "influxdb,/var/lib/influxdb\n",
      "zabbix,/var/lib/zabbix\n",
      "grafana,/usr/share/grafana\n",
      "logstash,/opt/logstash\n",
      "sphinx,/var/lib/sphinx\n",
      "redis,/var/lib/redis\n",
      "hacluster,/var/lib/heartbeat/cores/hacluster\n",
      "ricci,/var/lib/ricci\n",
      "nrpe,/var/run/nrpe\n",
      "telegraf,/etc/telegraf\n",
      "test,/home/test\n",
      "cloud-agent,/opt/cloud-agent/agent\n"
     ]
    }
   ],
   "source": [
    "echo i love you | wc -l\n",
    "echo i love you | wc -c\n",
    "echo i love you | wc -w\n",
    "\n",
    "#将当前文件夹内所有文件的名字中得所有空格替换为_。其中g代表所有，\n",
    "#rename 's/ /_/g' *  \n",
    "\n",
    "# ubuntu脚本apt-get安装\n",
    "#echo 'e' | sudo -S apt-get -y install $i\n",
    "\n",
    "#改变read的分隔符且不影响循环内部的IFS\n",
    "while IFS=: read user pass uid gid fullname homedir shell; do\n",
    "    echo $user,$homedir; \n",
    "done < /etc/passwd\n",
    "\n",
    "exec 2> /tmp/$0.log     #重定向shell本身的标准错误\n",
    "exec 3 < /tmpfile        #打开新文件描述符3\n",
    "read name < &3            #从3指向的文件读取\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
